Fix Git ref refresh resource storms by juliusmarminge · Pull Request #4727 · pingdotgg/t3code

juliusmarminge · 2026-07-28T11:02:17Z

What changed

remove the five-second client-side ref polling loop and refresh once per connection generation
expire idle ref atoms after 30 seconds
key repository/ref snapshots by the common Git directory
coalesce duplicate refreshes, single-flight full scans, and bound Git subprocess concurrency
replace separate local/remote ref enumeration with one for-each-ref call
invalidate cached snapshots after ref mutations

Root cause

Each retained vcs.listRefs atom polled independently. On this repository that multiplied 31 concurrent requests into repeated full scans across 838 worktrees and roughly 16,700 refs. The server had no shared snapshot, single-flight guard, or concurrency bound, so identical requests launched many expensive git branch, git worktree list, and ref-enumeration processes at once.

Impact

Ref lists remain fresh on connection changes and explicit refreshes, while background idle work no longer creates a Git subprocess storm. Mutations still invalidate snapshots immediately.

Verification

34 focused client/server tests
targeted formatting, lint, and affected-package typechecks
integrated web verification: at most two expensive Git children and one worktree scan
integrated mobile idle verification: zero expensive Git samples across 60 settled samples

Co-authored-by: codex codex@users.noreply.github.com

Note

Fix Git ref refresh resource storms with coalesced snapshots, exponential backoff, and invalidation hooks

Replaces periodic polling for VCS refs with a single refresh per connection generation, retried on failure using exponential backoff (1s base, 30s cap), and a 30s idle TTL instead of 5 minutes.
Adds a coalesced, cached listRefs snapshot in GitVcsDriverCore so concurrent paginated calls share one underlying for-each-ref scan, with generation tracking to prevent stale cache reuse.
Invalidates cached VCS refs (both in-memory and persisted) after any ref-affecting mutation (pull, push, createRef, switchRef, worktree ops, publish, etc.) on both success and failure via new onSettled hooks on RPC commands.
Adds removeVcsRefs and clearVcsRefs to EnvironmentCacheStore (web IndexedDB, mobile SQLite) and a per-environment persistence lock to serialize cache mutations.
Fixes branch list pagination in BranchToolbarBranchSelector to only trigger next-page loads on downward scroll past a threshold, not on layout or array length changes.
Risk: Idle list-refs atoms now expire after 30s instead of 5 minutes, causing more frequent re-fetches when the atom is re-subscribed after a period of inactivity.

^{Macroscope summarized 2c4c29c.}

Note

Medium Risk
Large changes to GitVcsDriverCore caching, invalidation, and client ref streams affect branch lists and worktree-heavy repos; behavior is heavily tested but touches core VCS paths.

Overview
Stops Git ref refresh from spawning subprocess storms by replacing per-atom 5s polling with one refresh per connection generation (exponential retry, 30s idle atom TTL) and a server-side coalesced snapshot keyed by common Git dir (for-each-ref + worktree metadata instead of separate branch listings).

Invalidation: VCS mutations (pull, worktree, create/switch ref, publish, stacked actions) clear persisted vcs-refs via new clearVcsRefs / removeVcsRefs on web and mobile stores, bump a per-environment revision, and invalidate server snapshots after Git-changing RPCs; listRefs accepts optional refresh: true (e.g. PR local branch lookup).

UI: Branch picker pagination loads the next page only on downward scroll near the bottom, with a dedicated isFetchingNextPage flag so background refreshes are not shown as “loading more.”

^{Reviewed by Cursor Bugbot for commit 2c4c29c. Bugbot is set up for automated code reviews on this repo. Configure here.}

coderabbitai · 2026-07-28T11:02:29Z

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bf4cae2f-84f1-4725-a1b6-03f7acf28615

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch t3code/git-ref-refresh-resource-storm

_{Comment @coderabbitai help to get the list of available commands.}

macroscopeapp · 2026-07-28T12:08:02Z

Approvability

Verdict: Needs human review

This PR introduces substantial new caching infrastructure for Git ref operations, including multi-layer caches with TTL management, coalescing of concurrent requests, environment-wide invalidation on mutations, and exponential backoff for failures. The complexity of the new state management and fundamental changes to ref fetching behavior warrant human review.

No code changes detected at 2c4c29c. Prior analysis still applies.

^{You can customize Macroscope's approvability policy. Learn more.}

Co-authored-by: codex <codex@users.noreply.github.com>

cursor

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

^{Bugbot Autofix is ON, but a cloud agent failed to start.}

^{Reviewed by Cursor Bugbot for commit 2c4c29c. Configure here.}

@juliusmarminge

## What's Changed * Add OTA update checks to mobile settings by @juliusmarminge in pingdotgg/t3code#4686 * fix(mobile): threads load snapped to bottom on iOS by @t3dotgg in pingdotgg/t3code#4689 * feat: default sidebar v2 on for nightly and dev builds by @t3dotgg in pingdotgg/t3code#4491 * fix(web): 33 web UI fixes by @maxktz in pingdotgg/t3code#4700 * Make mobile Thread List v2 the default by @juliusmarminge in pingdotgg/t3code#4717 * Fix mobile showcase workflow without Clerk by @juliusmarminge in pingdotgg/t3code#4718 * Fix relay credential lookup for unlinked environments by @juliusmarminge in pingdotgg/t3code#4692 * Settle merged PR threads immediately by @t3dotgg in pingdotgg/t3code#4704 * feat(web): add maria's sidebar header artwork toggle by @maxktz in pingdotgg/t3code#4652 * feat(web): add appearance settings category by @maxktz in pingdotgg/t3code#4715 * fix(desktop): allow updater-controlled relaunch by @0x4bs3nt in pingdotgg/t3code#4721 * fix(web): prevent diff panel scroll jumping by @0x4bs3nt in pingdotgg/t3code#4724 * Link inline code file paths in chat markdown by @juliusmarminge in pingdotgg/t3code#4726 * Fix Android showcase capture and rebuild v2 queued rows by @juliusmarminge in pingdotgg/t3code#4730 * perf(server): negotiate permessage-deflate on the websocket by @t3dotgg in pingdotgg/t3code#4705 * docs: overhaul agent guidance by @t3dotgg in pingdotgg/t3code#4782 * Prevent sidebar row labels from truncating by @juliusmarminge in pingdotgg/t3code#4789 * perf(server): gzip large thread snapshots by @t3dotgg in pingdotgg/t3code#4788 * fix(web): stashed prompts now survive switching providers by @t3dotgg in pingdotgg/t3code#4787 * Fix Git ref refresh resource storms by @juliusmarminge in pingdotgg/t3code#4727 * perf(server): trim stale context-window rows and drop dead replay RPC by @t3dotgg in pingdotgg/t3code#4791 * fix(web): defer command palette filesystem navigation by @juliusmarminge in pingdotgg/t3code#2109 * fix(release): skip scripts during Vercel installs by @t3dotgg in pingdotgg/t3code#4796 * refactor(client): share filesystem browse navigation by @juliusmarminge in pingdotgg/t3code#4797 * fix(mobile): defer filesystem navigation by @juliusmarminge in pingdotgg/t3code#4799 * refactor(server): use native HTTP compression streams by @juliusmarminge in pingdotgg/t3code#4798 * Remove Connect waitlist and add GA announcement tooling by @juliusmarminge in pingdotgg/t3code#4691 **Full Changelog**: pingdotgg/t3code@v0.0.29...v0.0.30 ## What's Changed * Add OTA update checks to mobile settings by @juliusmarminge in pingdotgg/t3code#4686 * fix(mobile): threads load snapped to bottom on iOS by @t3dotgg in pingdotgg/t3code#4689 * feat: default sidebar v2 on for nightly and dev builds by @t3dotgg in pingdotgg/t3code#4491 * fix(web): 33 web UI fixes by @maxktz in pingdotgg/t3code#4700 * Make mobile Thread List v2 the default by @juliusmarminge in pingdotgg/t3code#4717 * Fix mobile showcase workflow without Clerk by @juliusmarminge in pingdotgg/t3code#4718 * Fix relay credential lookup for unlinked environments by @juliusmarminge in pingdotgg/t3code#4692 * Settle merged PR threads immediately by @t3dotgg in pingdotgg/t3code#4704 * feat(web): add maria's sidebar header artwork toggle by @maxktz in pingdotgg/t3code#4652 * feat(web): add appearance settings category by @maxktz in pingdotgg/t3code#4715 * fix(desktop): allow updater-controlled relaunch by @0x4bs3nt in pingdotgg/t3code#4721 * fix(web): prevent diff panel scroll jumping by @0x4bs3nt in pingdotgg/t3code#4724 * Link inline code file paths in chat markdown by @juliusmarminge in pingdotgg/t3code#4726 * Fix Android showcase capture and rebuild v2 queued rows by @juliusmarminge in pingdotgg/t3code#4730 * perf(server): negotiate permessage-deflate on the websocket by @t3dotgg in pingdotgg/t3code#4705 * docs: overhaul agent guidance by @t3dotgg in pingdotgg/t3code#4782 * Prevent sidebar row labels from truncating by @juliusmarminge in pingdotgg/t3code#4789 * perf(server): gzip large thread snapshots by @t3dotgg in pingdotgg/t3code#4788 * fix(web): stashed prompts now survive switching providers by @t3dotgg in pingdotgg/t3code#4787 * Fix Git ref refresh resource storms by @juliusmarminge in pingdotgg/t3code#4727 * perf(server): trim stale context-window rows and drop dead replay RPC by @t3dotgg in pingdotgg/t3code#4791 * fix(web): defer command palette filesystem navigation by @juliusmarminge in pingdotgg/t3code#2109 * fix(release): skip scripts during Vercel installs by @t3dotgg in pingdotgg/t3code#4796 * refactor(client): share filesystem browse navigation by @juliusmarminge in pingdotgg/t3code#4797 * fix(mobile): defer filesystem navigation by @juliusmarminge in pingdotgg/t3code#4799 * refactor(server): use native HTTP compression streams by @juliusmarminge in pingdotgg/t3code#4798 * Remove Connect waitlist and add GA announcement tooling by @juliusmarminge in pingdotgg/t3code#4691 **Full Changelog**: pingdotgg/t3code@v0.0.29...v0.0.30 Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.30

…pstream's coalesced ref snapshots, PR pingdotgg#4727)

…m conflict (#165) Blind theirs on fork/tim dropped main's vcs resource-storm fix while leaving main's vcs tests, so integration Test failed. Restore main's VCS client/server files and prefer ours for those paths on future Tim rebases. Co-authored-by: T3 Code PR Stack <41898282+github-actions[bot]@users.noreply.github.com>

fix(stack): rejoin fork VCS/UI APIs after main pingdotgg#4727 restore

…orktree cleanup Join upstream Git ref-refresh resource-storm fixes with fork contracts (failureKind, commit signing, worktree cleanup RPCs, reuse-base-branch UI) instead of leaving tip-only fix(stack) recovery commits after Tim whole-file conflict policies.

…nd; green tip Bring Tim layer tip to typecheck green by joining main ref-refresh VCS client state with fork failureKind/worktree-cleanup contracts, restoring filterBrowseEntries/reuse-base-branch surfaces Tim dropped, and fixing ChatView/Board call-site type errors left by incomplete Tim joins.

…orktree cleanup Join upstream Git ref-refresh resource-storm fixes with fork contracts (failureKind, commit signing, worktree cleanup RPCs, reuse-base-branch UI) instead of leaving tip-only fix(stack) recovery commits after Tim whole-file conflict policies.

…nd; green tip Bring Tim layer tip to typecheck green by joining main ref-refresh VCS client state with fork failureKind/worktree-cleanup contracts, restoring filterBrowseEntries/reuse-base-branch surfaces Tim dropped, and fixing ChatView/Board call-site type errors left by incomplete Tim joins. (cherry picked from commit 0e24917)

…orktree cleanup Join upstream Git ref-refresh resource-storm fixes with fork contracts (failureKind, commit signing, worktree cleanup RPCs, reuse-base-branch UI) instead of leaving tip-only fix(stack) recovery commits after Tim whole-file conflict policies.

…nd; green tip Bring Tim layer tip to typecheck green by joining main ref-refresh VCS client state with fork failureKind/worktree-cleanup contracts, restoring filterBrowseEntries/reuse-base-branch surfaces Tim dropped, and fixing ChatView/Board call-site type errors left by incomplete Tim joins. (cherry picked from commit 0e24917)

Brings in 20 upstream commits including native resource diagnostics (pingdotgg#2679), event-driven Git ref invalidation (pingdotgg#4727), 300MB installer size reduction (pingdotgg#4824), thread-shell preservation while detail loads (pingdotgg#4830), HTTP compression, and the v0.0.30 release prep. Merge resolutions of note: - vcs.ts / ProcessDiagnostics / EventNdjsonLogger: upstream superseded our polling-interval, WMI, and log-batching fixes with better implementations - build-desktop-artifact: kept fork artifact naming + bundled Codex plugin extraResource on top of upstream's size optimizations - ChatView/ChatComposer: both feature sets kept (send-morph/reveal/hydration + thread-shell loading, sendDisabledReason, ThreadSyncStatusPill) - RpcAuthorization: added fork-only shell.revealPath scope - AGENTS.md: kept fork operational rules, took upstream's product sections - ProjectionPipeline: expiry timestamps via DateTime.nowUnsafe (wall clock) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Conflict resolutions and fork fixups: - apps/web/src/localApi.ts: upstream pingdotgg#2679 moved `server.*` and `shell.openInEditor` off the `LocalApi` host facade (localApi.test.ts now asserts they are absent). Took upstream's version and dropped the fork's `server` shim, repointing its one caller (SettingsPanels.refreshProviderInstance) at `callPrimaryRpc` directly. - apps/server/src/auth/RpcAuthorization.ts: upstream's new scope map is exhaustive over WsRpcGroup, so declare scopes for the fork's 15 PR-workspace git.* RPCs (reads -> orchestration:read, mutations -> orchestration:operate). - packages/client-runtime/src/state/shell-sync.test.ts: add the removeVcsRefs/clearVcsRefs stubs that two fork-added cache mocks were missing (pre-existing breakage from the previous merge's pingdotgg#4727). Verified: pnpm typecheck clean across all 15 packages; web (1666), server (1771), client-runtime (497) tests pass; lint clean.

…nd; green tip Bring Tim layer tip to typecheck green by joining main ref-refresh VCS client state with fork failureKind/worktree-cleanup contracts, restoring filterBrowseEntries/reuse-base-branch surfaces Tim dropped, and fixing ChatView/Board call-site type errors left by incomplete Tim joins. (cherry picked from commit 0e24917)

github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:XL 500-999 changed lines (additions + deletions). labels Jul 28, 2026

juliusmarminge marked this pull request as ready for review July 28, 2026 12:01

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

Comment thread packages/client-runtime/src/state/vcs.ts

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

Comment thread packages/client-runtime/src/state/vcs.ts

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 501a738 to f2449b5 Compare July 28, 2026 12:54

macroscopeapp Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 6caf609 to 45ac080 Compare July 28, 2026 13:44

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts Outdated

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 2a9d07e to dfc1459 Compare July 28, 2026 14:11

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts Outdated

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from dfc1459 to 561c035 Compare July 28, 2026 14:24

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts

Comment thread packages/client-runtime/src/state/vcsAction.ts Outdated

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 561c035 to 4f3a660 Compare July 28, 2026 14:44

macroscopeapp Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcsRefInvalidation.ts Outdated

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 4f3a660 to ff4df7f Compare July 28, 2026 15:16

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from ff4df7f to 4ba5ea3 Compare July 28, 2026 15:30

github-actions Bot added size:XXL 1,000+ changed lines (additions + deletions). and removed size:XL 500-999 changed lines (additions + deletions). labels Jul 28, 2026

macroscopeapp Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcsRefInvalidation.ts Outdated

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 4ba5ea3 to 25c0c7a Compare July 28, 2026 15:37

macroscopeapp Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread packages/client-runtime/src/state/vcs.ts

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts Outdated

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/server/src/vcs/GitVcsDriverCore.ts

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 25c0c7a to 7fa76c8 Compare July 28, 2026 15:44

juliusmarminge and others added 2 commits July 29, 2026 00:30

Gate ref pagination on user scroll

d82cc53

Co-authored-by: codex <codex@users.noreply.github.com>

Handle subpixel ref list scrolling

2c4c29c

Co-authored-by: codex <codex@users.noreply.github.com>

juliusmarminge force-pushed the t3code/git-ref-refresh-resource-storm branch from 6deb060 to 2c4c29c Compare July 28, 2026 22:31

cursor Bot reviewed Jul 28, 2026

View reviewed changes

Comment thread apps/web/src/state/paginatedBranches.ts

juliusmarminge merged commit 38a6e3c into main Jul 28, 2026
17 checks passed

juliusmarminge deleted the t3code/git-ref-refresh-resource-storm branch July 28, 2026 22:42

NateZwainleskPLA mentioned this pull request Jul 29, 2026

[Bug]: Local backend becomes CPU-bound and unresponsive during idle VCS/provider refreshes, causing reconnects #4773

Open

2 tasks

SergeSerb2 mentioned this pull request Jul 29, 2026

Port upstream thread-performance improvements (top 10 + fixes) SergeSerb2/SurgeCode#317

Merged

bzbetty mentioned this pull request Jul 29, 2026

[Bug]: Excessive CPU usage on Windows #4182

Open

2 tasks

haye-go mentioned this pull request Jul 29, 2026

perf: stop tracing every command-path probe and bound git spawns #4778

Open

darl added a commit to darl/t3code that referenced this pull request Jul 29, 2026

Merge upstream main (drop local listRefs snapshot cache in favor of u…

2c79291

…pstream's coalesced ref snapshots, PR pingdotgg#4727)

t3code-tarik02 Bot mentioned this pull request Jul 29, 2026

prepare stable release 2026.7.2900+tarik02 tarik02-org/t3code#84

Merged

patroza mentioned this pull request Jul 29, 2026

fix(stack): restore upstream Git ref refresh after Tim conflict patroza/t3code#165

Merged

3 tasks

patroza mentioned this pull request Jul 29, 2026

fix(stack): rejoin fork VCS/UI APIs after main #4727 restore patroza/t3code#166

Merged

5 tasks

patroza added a commit to patroza/t3code that referenced this pull request Jul 29, 2026

Merge pull request #166 from patroza/fix/layer-gates-green

c726700

fix(stack): rejoin fork VCS/UI APIs after main pingdotgg#4727 restore

t3code-tarik02 Bot mentioned this pull request Jul 29, 2026

prepare stable release 2026.7.3000 tarik02-org/t3code#95

Open

jakeleventhal mentioned this pull request Jul 29, 2026

Refresh working tree diff after turns #3906

Open

TheDarkPhantom mentioned this pull request Jul 30, 2026

chore: merge upstream/main (v0.0.31) TheDarkPhantom/avicode#49

Merged

aaditagrawal mentioned this pull request Jul 31, 2026

sync: port upstream websocket/compression and git perf work (#4291–#4985) aaditagrawal/t3code#191

Open

Uh oh!

Conversation

juliusmarminge commented Jul 28, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changed

Root cause

Impact

Verification

Fix Git ref refresh resource storms with coalesced snapshots, exponential backoff, and invalidation hooks

Uh oh!

coderabbitai Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Uh oh!

Uh oh!

Uh oh!

macroscopeapp Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Approvability

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

juliusmarminge commented Jul 28, 2026 •

edited by cursor Bot

Loading

coderabbitai Bot commented Jul 28, 2026 •

edited

Loading

macroscopeapp Bot commented Jul 28, 2026 •

edited

Loading